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Abstract 

We give a unified treatment of the convergence of random series and 
the rate of convergence of strong law of large numbers in the framework 
of game-theoretic probability of Shafer and Vovk [^J . We consider games 
with the quadratic hedge as well as more general weaker hedges. The lat- 
ter corresponds to existence of an absolute moment of order smaller than 
two in the measure-theoretic framework. We prove some precise relations 
between the convergence of centered random series and the convergence of 
the series of prices of the hedges. When interpreted in measure-theoretic 
framework, these results characterize convergence of a martingale in terms 
of convergence of the series of conditional absolute moments. In order to 
prove these results we derive some fundamental results on deterministic 
strategies of Reality, who is a player in a protocol of game-theoretic proba- 
bility. It is of particular interest, since Reality's strategies do not have any 
counterparts in measure-theoretic framework, ant yet they can be used to 
prove results which can be interpreted in measure-theoretic framework. 

Keywords and phrases: Kronecker's lemma, law of the iterated logarithm, 
Levy's extension of Borel-Cantelli lemma, Marcinkiewicz-Zygmund strong law, 
three-series theorem. 

1 Introduction 



In standard textbooks on measure-theoretic probability, the strong law of large 
numbers (SLLN) is proved using Kronecker's lemma. As a precondition for 
Kronecker's lemma, the convergence of a random series is usually stated in 
the form of three-series theorem. Game-theoretic counterpart in Section 4.2 of 
Shafer and Vovk ([H]) basically follows the same line of argument. However 
game-theoretic forms of various conditions for convergence of random series have 
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not been studied in detail. Indeed, game-theoretic counterparts of the standard 
three-series theorem have to be stated more carefully than in the measure- 
theoretic setting in several respects, such as the treatment of truncation of 
random variables and the martingale nature of game-theoretic framework. In 
particular we need to take into account Gilat's counter example ([3, see also 
[5], [3]) to three-series theorem for martingales. 

In this paper we give a unified game-theoretic treatment of convergence of 
random series and the rate of convergence of SLLN. We consider games with 
the quadratic hedge. For an i.i.d. case in the terminology of measure-theoretic 
probability, the law of the iterated logarithm (LIL) gives the precise rate. In 
game-theoretic probability which corresponds to a non-identical case, the rate 
of convergence may be slower. We give the precise rate in the game-theoretic 
framework. 

We also consider games with more general weaker hedges. Marcinkiewicz- 
Zygmund strong law ([H], |H], [13]) suggests that the rate of convergence of 
random series and SLLN should depend on the existence of moments. In Section 
[5] we will show that the rate is determined by the inverse function of the hedge 
function. 

In order to derive results on convergence of random series, we study some 
topics. One topic is the set of convergence of a martingale. If a capital process is 
required to be non-negative, the convergence theorem for a non-negative capital 
process stated in [SI] is sufficient. However it is useful to consider the set of 
convergence for an arbitrary capital process which may be negative. 

Another topic is deterministic strategies of Reality. We propose a notion 
called "compliance" concerning deterministic strategies of Reality and prove 
some results on them. In [24], Reality's strategy is only briefly discussed. Fur- 
thermore only randomized strategies of Reality are considered in Section 4.3 of 
[24] and Section 7 of [13] . Our deterministic strategies of Reality can be under- 
stood as "derandomizations" of the randomized strategies in [24] and [13] . It is 
of interest that deterministic strategies of Reality, which do not have any coun- 
terparts in measure-theoretic probability, can be used to prove results which 
can be interpreted in measure-theoretic probability. 

The organization of this paper is as follows. In Section [5] we consider sets of 
convergence in the bounded forecasting game and establish preliminary results 
on the implication of the convergence of a random series to the convergence of 
the series of prices for Reality's moves. We also treat the coin-tossing game 
as a special case of the bounded forecasting game. In Section [3] we consider 
bounded forecasting game with quadratic hedge and prove various results on 
convergence of random series. In Section |4] we study the rate of convergence of 
SLLN in unbounded forecasting game with quadratic hedge and in Section Owe 
generalize our results to games with more general weaker hedges. We end the 
paper with some discussion on further topics in Section [5] 
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2 Preliminary results for bounded forecasting 
game and the coin-tossing game 

In this section we consider the bounded forecasting game of Section 3.3 of [M] 
and the coin-tossing game as a special case. This section also serves as a brief 
introduction to game-theoretic probability. 

2.1 Definitions and some notions 

Consider a perfect information game among three players: Forecaster, Skeptic 
and Reality. Let C > be given. Before the start of the game, Skeptic an- 
nounces his initial capital JC^ = D > Q. K.q = 1 va Section 3.3 of but 
in this section, for our discussion of the bounded forecasting game, it is more 
convenient to let Skeptic announce his initial capital /Cq = D. Then, at each 
round n = 1, 2, . . . , of the game, these players announce their moves in the or- 
der: Forecaster, Skeptic and Reality. At each round, Forecaster first announces 
m„, which is interpreted as the price for Reality's move x„. Forecaster has to 
announce a coherent price m„ (Section 1.2 of 24), that is, with the announced 
price m„. Reality should always be able to prevent Skeptic from strictly in- 
creasing his capital /C„ for sure. Given the price. Skeptic then announces the 
amount Af„ he bets. Finally Reality announces her move Xn € [— C, C]. The 
payoff to Skeptic at the n-th round is Mn{xn — rUn) and his capital is updated 

as ICn = K-n-l + Mn{Xn - m„). 

More precisely, the protocol of bounded forecasting game is written as fol- 
lows. 

Bounded Forecasting Game 

Parameter: C > 

Players: Forecaster, Skeptic, Reality 

Protocol: 

Skeptic announces his initial capital )Cq = D > 0. 
FOR n = 1,2,...: 

Forecaster announces m„ e [— C, C]. 
Skeptic announces M„ e M. 
Reality announces Xn € [— C, C]. 
K-n := ICn-i + Mn{xn - rn„). 
Collateral Duties: Skeptic must keep /C„ non-negative. Reality 
must keep /C„ from tending to infinity. 

Note that Forecaster's price to„ G [— C, C] is clearly coherent, because Re- 
ality can always choose Xn — run, so that ICn = ICn-i irrespective of M„. 

A strategy V = {7^n}n>i of Skeptic specifies M„, > 1, in terms of past 
moves of other players m^, a;fe, fc = l,...,ri— 1, and the current price m„: 

Mn = 7'„(mi,xi, . . . ,m„„i,a;„_i,m„). 
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We define the capital process KT = {JC^}n>Q for a given strategy V recursively 
by ICq ~ and 

/C,^ = /C^_i +Vn{rni,xi, . . . , m„_i, a;„_i, m„)(a:„ - m„), n>l. 

iC^ is the cumulative payofi^ to Skeptic up to round n under the strategy P 
(without the initial capital /Co ~ D). With our definition, IC^ = is dis- 
tinguished from the initial capital JCq — D announced by Skeptic. We call a 
sequence of real- valued functions Snirni, xi, . . . , x„, m„) of mi, xi, . . . , ran, Xn, 
n > 0, a capital process if 5„ = /C^ for some strategy V. 

An infinite sequence ^ = (toi, xi, m2, X2, . . . ) of moves of Forecaster and 
Reality is called a path. Define the sample space 

5 = = (mi, xi, 7712, X2, ■ ■ ■) \ rnn,Xn e [-C, C], Vn > 1} 

as the set of paths. We regard JC^ and V as functions of ^. They are denoted 
by /C^(C) and VniO^ which actually depend only on prefixes of ^ of length 2n 
and 2n — 1, respectively. 

Any subset of S is called an event. We say that Skeptic can force E if 
there exists a strategy V of Skeptic, such that 

/cr(0>-l, VeeS,Vn>0, (1) 

and 

E ^ limsup/C^(^) = oo. 

n 

In this paper we do not make the distinction between forcing (lim„/C^ — oo) 
and weak forcing (limsup„/C^ = cx)) in view of Lemma 3.1 of 24 . A strategy 
V satisfying ([T]) is called prudent, i.e., if Skeptic observes his collateral using V 
with the initial capital /Cq = 1. 

For two events Ei,E2 C S, the event Ei =J> E2 stands for i?f U E2, where 
E^ is the complement of Ei. Ei ^ E2 stands for both implications: 

{E^ U E2) n {El U E^) = {El n E2) U {E^ n E^) = {EiAE2f, 

where A denotes the symmetric difference. Note that Ei E2 and E'p E2 
are the same as a subset of 5. In view of Lemma 3.1 of [H], Skeptic can force 
El <^ E2 if and only if Skeptic can force both Ei E2 and E2=> Ei. 

2.2 A set of convergence 

Martingale convergence theorems in measure-theoretic probability state that 
the limit of a martingale exists and is finite almost surely if the martingale is 
bounded in . In Section VII. 5 of [27] the set of convergence was studied when 
the condition is not satisfied. 

Game-theoretic probability also has convergence theorems. If a capital pro- 
cess is required to be non-negative, the convergence theorem holds. However it 
is useful to consider a strategy whose capital process may be negative in order 
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to construct a strategy whose capital process is non-negative as we will do later. 
Then we will prove a game-theoretic version of a simple case of results in |27j . 
The results are used in a later section. 

Let P be a strategy of Skeptic. Denote the minimum possible gain L„ = 
Lni'mi, xi, . . . , m„_i, Xn-i, rUn) to Skeptic at the round n (after he knows Fore- 
caster's move TO„) under V by 

L„ = min 7'„(mi, xi, . . . , m„_i, x„_i, m„)(a; - m„) 

x£l-C,C] 

yVn X (C — nin) otherwise. 

For D > 0, we define the stopping time = t^{£,) as the first time D + IC^ 
may be negative: 

= min{n > 1 : /Cf > . . . , /C^_i > -D, KZ_^ + L„ < -D). 

As usual = +00 if the set on the right-hand side is empty. The truncation 
of V at the loss —D is defined as 

Vn{mi,xi, . . . ,mn) a n< T-p 
otherwise. 

Note that starting with the initial capital of £> > 0, Skeptic observes his collat- 
eral duty by employing , i.e., D + ICI^ is always nonnegative. 
We now prove the following proposition. 

Proposition 2.1. Let V he any strategy in the hounded forecasting game. Let 
denote the event 

B'^ = {Vn IS hounded } = {C| sup \Vn{0\ < 00}. (2) 

n 

Skeptic can force 

B^ (/C^ converges in or (lim sup = +00 and liminf /C^ = — 00)). 

n n ' 

(3) 

By "convergence in R" we mean that lim„ /C^ exists and is finite. In later 
statements we will omit "in R" for simplicity. Proposition 12 . II means that given 
any strategy V, there exists another prudent strategy Q of Skeptic, such that 
limsup„/C,^ = 00 if is violated. 

Proof. Note that the convergence or divergence of /C^ is classified into the fol- 
lowing five exclusive cases: 

(i) —CO < liminf„/C^ ~ limsup„/C^ < 00 (convergence), 

(ii) —00 < liminf„/C^ < limsup„/C^ < 00 (bounded oscillation). 



Vnimi,xi, . . . ,m„) 
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(iii) — oo < liminfji JC^ < limsup„ JC^ — oo, 

(iv) — oo = liminfn /C^ < limsup„ /C^ < oo, 

(v) — oo = liminf„ /C^, lim sup„ /C^ = oo (two-sided unbounded oscillation). 

According to this classification, the sample space S is partitioned into five sub- 
sets , . . . , . By Lemma 3.2 of 24 , it suffices to construct a prudent strat- 
egy Q of Skeptic for each of the cases Ef, i — 2,3, 4, such that G n Ef 
implies limsup„ /C® = oo, i = 2, 3, 4. 

We consider the case (iii) in detail. As noted above, for each D > 0, D + lC^ 
is always nonnegative. Consider dividing the initial capital of 1 into countably 
infinite accounts with initial capitals l/2-fl/4-|--- - — 1. For the I?-th account 
with the initial capital of 1/2^, we apply the strategy / {D2^). The resulting 
combined strategy Q is written as 

oo 

Q = V -^V" 
^ ^ D2d' ■ 

D = l 

Then the capital process of Q is written as 

oo oo -. oo -. 

1 + - E i + E i^^r = E + 

15=1 15=1 15=1 

and hence Q is prudent. Now for each ^ G B^OE^, there exist positive constants 
Di = Di (0,-02 = D2iO, such that 

Then since |m„|, \xn\ < C 

|L„(C)| < 2Ci?2(C), Vn>l. 
Consider D > Di{(,) + 2CD2{0- Then for aU n > 1, we have 
ICliC)>~Di{0>~D, k = l,...,n-l, 

and 

/C,^_i(0 + L„(0 > -D,{0 - 2CD2{0 > -D. 

Hence for this D we have t^{C} — oo and /C^ = /C^ , Vn > 1. Therefore 
^ e B^ n i?3 =4> limsup„/C^(0 = oo. This proves the case of (iii). 

The case (iv) is proved by the symmetry of the bounded forecasting protocol, 
i.e. by considering —V instead of V . 

Finally (ii) can be proved by the standard argument involving Doob's up- 
crossing lemma (see Lemma 4.5 of 124^). Note that Lemma 4.5 of [23] is for 
the case of prudent V. In our case, V is not necessarily prudent. However 
again combining truncations , D = 1,2,..., with the argument of upcross- 
ing lemma, we can construct a prudent Q such that limsup„/C^(0 = oo for 
each ^ € B^ n E2 ■ This proves the proposition. □ 
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We call a strategy V uniformly cautious if 



sup |7'„(0I < oo. 

5GH,n>l 

For uniformly cautious V, = S. Therefore if V is uniformly cautious, then 
Skeptic can force the right-hand side of ([3]). The reason for considering 
in ^ and uniformly cautious strategies is that they eliminate doubling type 
strategies. 

Using Proposition 12 .11 we can prove the following result, which is a general- 
ization of results in Section 2.2.2 of pH] . 

Proposition 2.2. In the bounded forecasting game Skeptic can force 



Xn converges 



converges or 



(lim sup TOfc 



fc=i 



-oo and lim mf > rrifc 

n ^ — ^ 

fe=l 



-oo) (4) 



converges 



E 



a;„ converges or 



(lim sup y^ Xfc ^ -|-oo and liminf a:^: = — oo) (5) 



fe=i 



fc=i 



Proof. Let y„ = X]fe=i(^*: ~ ?Tife)- Consider the uniformly cautious strategy V 
such that Mn = 1. Then /C^ = 1^. By Proposition [2TT1 Skeptic can force 

Yn converges or (lim sup y„ = -|-oo and liminf y„ = — oo). 



We separate the sums in as Yn — ^2tk—i — ^A;— i and restrict relevant 
events to the particular event Eq ~ {X^n converges}. Then clearly Skeptic 
can force (O is proved similarly, by switching the roles of Xn and m„. □ 



2.3 Some applications 

Consider the multi-dimensional bounded forecasting game (cf. [14]) defined as 
follows. Reality's move space is a compact set X of R"*. Forecaster's move space 
is the convex hull co(A') of X and Skeptic's move space is W^. Denote the moves 
by Forecaster, Skeptic and Reality by /Lt„, m„ and respectively. The payoff 
to Skeptic is vein ■ (Xn — fJ-n), where "•" denotes the standard inner product in 
R'^. The protocol of the multi-dimensional bounded forecasting game is written 
as follows. 



Multi-dimensional Bounded Forecasting Game 
Parameter: a compact region X cM."^ 
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Players: Forecaster, Skeptic, Reality 
Protocol: 

Skeptic announces his initial capital ICq — D > 0. 
FOR n = 1,2,...: 

Forecaster announces /x„ € coA". 

Skeptic announces tn„ G M'^. 

Reality announces Xn G 

Collateral Duties: Skeptic must keep /C„ non-negative. Reality 
must keep /C„ from tending to infinity. 

It is easily seen that ProDOsition l2 . 1 1 holds in the multi-dimensional bounded 
forecasting game. 

We now consider the coin-tossing game and give a game-theoretic probability 
version of Levy's extension of Borel-Cantelli lemma. The protocol of the coin- 
tossing game is written as follows. 

Coin-tossing game 
Protocol: 
/Co = l. 

FORn= 1,2,...: 

Forecaster announces Pn G [0,1] 
Skeptic announces Af„ G M. 
Reality announces a;„ G {0, 1}. 

ICn ■= ICn-l + Mn{Xn - Pn)- 



We have the following. 

Example 2.3. (A game-theoretic version of Levy's extension of Borel-Cantelli 
lemma) In the coin-tossing game Skeptic can force 

< oo ^a;„ < oo. (6) 

n n 

Remark 2.4. The statement may be easier to understand if we rewrite it as 
follows: Skeptic can force 

< oo => x„ < oo 

n n 

and 

y^Pn = oo=»y^a:n = oo. 

n n 

Coin-tossing game is a special case of the bounded forecasting game (with 
C = D — 1), in such a way that the move space of Reality is restricted to {0, 1} 
and the move space of Forecaster is restricted to [0, 1] by coherence. Therefore, if 
Skeptic can force an event E in the bounded forecasting game, then Skeptic can 
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force E in the coin-tossing game. In the coin-tossing game, J2n^n and J2nPn 
are non-negative series and they either converge to finite values or diverge to 
+00. Therefore the case of two-sided unbounded oscillation on the right-hand 
side of (HI and ([5]) is impossible, which implies that Skeptic can force 

Levy's extension of Borel-Cantelli lemma in measure-theoretic probability is 
usually stated as follows (cf. Theorem 12.15 of [21] )■ 

Proposition 2.5. Let Xn, n — 1,2, ... , be a sequence of 0-1 random variables 
adapted to filtration {J-n}- Let pn = E{Xn \ J-n-i). Then almost surely 

(ii) E„P« = oo ^ linijv(E^Li2;"/E!LiPn) = 1- 

If we weaken (ii) to J^nP"- = co =^ J2n — oo, then the measure-theoretic 
extension looks similar to Theorem 12.31 However there are some basic differ- 
ences. In our setting, p„'s are only "prequentially" (eg. |30|) announced by 
Forecaster and there is no need to specify the full probability measure on a;„, 
n — 1,2, ... . Also, in measure-theoretic framework the null set, where these 
implications do not hold, may depend on the underlying probability measure. 
On the other hand, in the game-theoretic setting, we have constructed an ex- 
plicit strategy Q forcing ^ and the behavior of its capital process K.^ on the 
symmetric difference of two sets in (jG]) is explicitly understood. Furthermore in 
Proposition 14.81 of Section |3] we will strengthen the rate of convergence in (ii). 

3 Bounded forecasting game with quadratic hedge 

The standard measure-theoretic three-series theorem involves truncation of ran- 
dom variables and their means and variances. In considering game-theoretic 
counterpart of the standard setup, we here consider the simple case that the 
truncation is given before the game, i.e. we consider a variant of bounded fore- 
casting game. In addition we assume that the quadratic hedge is available to 
Skeptic. From now on for simplicity we assume JCq = 1. The protocol for this 
section is written as follows. 

Bounded Forecasting with Quadratic Hedge (BFQH) 
Parameter: C > 
Players: Forecaster, Skeptic, Reality 
Protocol: 
/Co = l. 

FOR n = 1,2,...: 

Forecaster announces m„ e [— C, C] and t>„ e [0, — rn^]. 
Skeptic announces M„ e M and V"„ G R. 
Reality announces x„ € [— C, C]. 

K-n := /C„-l + MniXn - TO„) + Vn{{Xn - TO„)^ - t>„). 

Collateral Duties: Skeptic must keep /C„ non-negative. Reality 
must keep from tending to infinity. 
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In this protocol the following theorem holds. 
Theorem 3.1. In BFQH Skeptic can force 

< CO "y^X^n — nin) converges. 



Before giving a proof of this theorem, we discuss some features of the theorem 
and the protocol BFQH. 

The implication "X^n < oo J^ni^n ~ converges" holds even for 
the case of C = +00, but the the converse implication does not hold for C = 
+00, as shown in Lemma 14.31 below. Therefore the main point of Theorem 13. II 
is that Skeptic can force the equivalence of both sides in the case < C < +00. 

In BFQH the ranges of Vn and Vn are different from both the bounded 
forecasting game in Section 3.1 of [23] and the unbounded forecasting game in 
Section 4.1 of [23]. First, we allow Vn to be negative. In the usual unbounded 
protocol, if Skeptic announces negative Vn then it violates his collateral duty 
because /C„ —00 as x„ -> 00. However this does not happen in the above 
protocol because |x„| < C is bounded. Second, we restrict w„ £ [0, — m^] for 
the game to be coherent. For example, consider the case r7i„ — 0, Vn — + e 
(e > 0), Mn = and Vn = -1, Then /C„ = /C„_i - xl + + e > /C„_i + e for 
all Xn € [—C, C]. More precisely we should restrict m„ and w„ such that 

zdpn and w„ = / (z - m„)^dp„ (7) 

[-ac] Ji-c,c] 

for some probability measure p„. For u„ £ [0, — rn^J it is easily checked that 
([7]) holds for the two-point measure 

,r 1 - JUn/C 1 + m„/C 
Pn({an}) = and p„({t)„}) , 



where a„ = m„ + (-C - mn)^J^^^ and fe„ m„ + (C - mn)^J^^^. 

BFQH can be regarded as a variant of the two-dimensional bounded forecast- 
ing game in Section [2.31 with an additional restriction at each round to Reality. 
In the multi-dimensional bounded forecasting game let X = [— 2C, C] x [0, 4C^], 
^J■n ^ {mn, Vn), m„ = (M„, K), Xn = (xn, (xn - m„)2). Also at each round of 
the game put an additional restriction to Reality's move space depending on 
Forecaster's move as Xn G = {{x, {x — m„)^)| — C < x < C}. Since the 
restriction is advantageous to Skeptic, Proposition 12. II holds for BFQH. 

Now we give a proof of Theorem 13.11 We use the notation 

n n 00 

Yn = ^(Xfc - mfc), = 0, An^Y Vk, Aoo = ^ Vk- 
k=l k=l k=l 

Proof of Theorem \3.1\ (=>) Consider a capital process 

n n 

Tn = Y^ - 2^yfc_i(xfc - ruk) + '^{{xk - irikf - Vk), 

k=l k=l 
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for Skeptic's strategy M„ = 2F„_i and Vn = 1. An is the compensator for Y^. 
Then by Lemma 4.6 and Lemma 4.7 of [2T, Skeptic can force that Yn converges. 

(<^=) Although the argument for this implication is essentially the same as the 
first part, we can not directly apply Lemma 4.6 of [23]. We prove this implication 
by Proposition 12. II Consider Skeptic's strategy 

Po-.Mn^ -2Yn-l, Vn=-l< 0, 

which is the negative of the above strategy. The capital process of Vq is given 
as /C^" — An—Y^. Note that B'^'^ of Proposition 12 . 1 1 for this strategy is the set 
of paths such that {Yn} is bounded. Therefore by a multi-dimensional version 
of Proposition 12.11 Skeptic can force 

{Yn} is bounded (A,i — Y^ converges or 

(limsup(A„ - Y^) = +00 and liminf(A„ - y^) = -00)). 

(8) 

By assumption Yn converges. Then {Yn} is bounded. Also Yn converges. Fur- 
thermore since An is non-decreasing, the second case of the right-hand side 
of ([H]) is impossible. Therefore Skeptic can force the event: Yn converges 
^ Aoc < 00. □ 

In Theorem 13.11 we considered the convergence of the series ~ 
In standard measure-theoretic probability theory, in the bounded case, the se- 
ries is split as X]n(^" ~ "^") — Sn^n ~ X^n™"' Then the convergence of 
X]n2;„ is discussed in terms of convergence of X)n ''^k ^^"^ X^n^"- Two-series 
theorem (see e.g. [27]) says that, under the assumption of independence, ^„ Xn 
converges if and only if '^n ^" converges and ^„ m„ converges. However, with- 
out the assumption of independence, the case where ^„ m„ does not converge 
and Xn'*'™ = 00, is very subtle. Indeed, Gilat [7] showed a counter example 
which implies that there exists no condition on the conditional expectations and 
conditional variances of a series of uniformly bounded random variables which 
is necessary and sufficient for the sum to converge almost surely. We give a 
game-theoretic version of Gilat 's result. 

Theorem 3.2. In BFQH there is no set D of infinite sequences of Forecaster's 
moves such that Skeptic can force 

{mi,vi,m2,V2, ■ ■ ■) E D a;„ converges. (9) 

n 

First we explain the intuition behind the proof. The following is Gilat 's 
example. Let {Zn} be the sequence of independent random variables with 

P{Zn - = i = P{Zn = -n^l^} for n > 1, 
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and Zq = 0. Then Z„ — t' and J2n does not converge almost surely. Let 
Xn = Zn — Zn-i and Yn = —Zn — Z^-i for n > 1. Then the conditional means 
and the conditional variances of Xn are the same as those of Yn . Furthermore 
J2k=i = ^,1^-0 and J22=i ^fc = ^2 J^kZl - Zn does not converge almost 
surely. 

We want to show that a strategy forcing © yields a contradiction. Intu- 
itively, if Skeptic can force an event E, then a "typical" or "random" sequence 
should satisfy E. Pick up a typical realization {ijn} of {Zn}- Then ry^ — >■ 
and r]n does not converge. The sequence = {rjn — Vn-i} is also typical 
for {Xn}- Then X]fc=i ^fe = Vn ^ and (mi, wi, m2, U2, • • •) G Further- 
more {un} = {—Vn — Vn-i} is typical for {Yn}- Then J2n 2/" does not converge 
and (toi, wi, m2, W2, • ■ •) ^ D- This is a contradiction. Since Forecaster knows 
the previous moves of Reality, the first randomness does not imply the second 
and third randomness in game-theoretic probability. Hence we construct three 
strategies and pick up a random sequence for the strategies. 

Proof- Assume that there exists a set D and a (prudent) strategy Q by which 
Skeptic can force that 

(toi, wi, TO2, W2, . . .) G -D S'n converges. (10) 

We prove that this assumption yields a contradiction. 
Also fix a (prudent) strategy V of Skeptic that forces 

Vn < oo <^=4> ^^(a;„ — m„) converges. (11) 

n n 

Such a strategy exists by Theorem 13. II 

Let a € N be a positive constant large enough to satisfy < (a — 1)^^/^ + 
a-i/2 < (J ajiji define 

w„ = (rt + a)^^, n > 1. 
Then Lol/^^ + iM"^ < (a - l)-i/2 + a-^/^ < C. Hence 

a;,y^ < C and a;„_i + a;„ < C^. 

We construct two strategies Qi and Q2 for Skeptic from Q. At round n. 
Skeptic by Q announces and These values depend on the moves of 

Forecaster and Reality until the previous round and the moves of Forecaster at 
the current round. So we write 

= M^(TOi,z;i,a;i,...,m„_i,w„_i,x„_i,m„, ■;;„), 

= y„^(r7ii,-!;i,a;i,...,m„_i,w„_i,a;„_i,m„,i;„). 

Define Q\ and Qi by 

AfSi =M^(-ryo,a;i,77i - rjo, -t]i,uj2,V2 - ?7i, . . . , -77n-i, w„), 

=V;^(-'70, Wi,?7i - T]Q,-'qi,UJ2,V2 - Vl, ■ ■ ■ ,-Vn-l,UJn), 

M^^ = - M^(-77o, wi, -?7i - 770, -?7i,a;2, -772 - 772, ... , -?7„_i, w„), 
K?" =K?(-'7o,wi,-77i - rio,-Vi,^2,-m -??2,...,-?7ri-i,w„). 
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where 770 = and |77„| — LUn"^. The sign of ??„ wih be defined below in p2)) . Since 
the sign of rjn is not defined yet, Qi and Q2 are not defined yet. However, we 
have -rjn^i G [-C,C], a;„ G [0,C'^ -r]l_;^] = [0,C^-u}n-i\ and |±7?„-77„_i| < 

Vn + Vn-i — ^11^ + ^ C*- Hence, if the sign of rjk is defined for each k < n, 
then M^i , l/„2i , M^^ and l^^^ are defined, because and y„2 are defined 
for m„ G [-C, C], t)„ G [0, - m^J and x„ G [-C, C]. 

The strategies of Qi and Q2 are explained as follows. Qi moves as Q moves 
when Forecaster's moves are specified as 

and Reality announces r\n — r\n-\- Similarly Q2 moves as Q when m„ = —rjn-i, 
Vn = uJn and Reality announces — -1 with the minus sign in the definition 
of 

Define a Skeptic's strategy T by T = {V + Qi + Q2)/3 and define (the sign 
of) r]n, n > 1, by 

^oj}/'^ ifA/J<0 
otherwise. 



vn = r\/2 :i_T (12) 



We claim that (for a given V) T and 7y„ are defined by mutual induction. Note 
that 770 is already defined as 0. At round n, rjk has been defined for each k < n. 
Then M^^ and are defined. Hence is defined. It follows that r]n is 

defined. By induction T and 7y„ are defined. 

The capital process /C„ is completely determined by strategies for Forecaster, 
Skeptic and Reality. By K.^''^-^ we denote the capital at round n when Fore- 
caster uses the strategy J^, Skeptic S and Reality TZ. In this proof we use the 
following strategies for Forecaster: 



and for Reality: 



J^2 ■ mn = -rjn-l, Vn = UJn, 

1Z2 : Xn = rjn - Vn^l, 
Tlz-Xn^ -tin - rjn^l. 

We claim m„ G [— C, C], v„ G [0, — m^] and a;„ G [— C, C] for each case. 
For J^i, t;„ = < and w„ G [0,C^] = [0,C^ - m^J. For ^"2, "4 = 
ri^_i = w„_i < and m„ G [— C, C], u„ = < — w„-i = — m'^ and 
f„ G [0, C2 - m^]. For Tei, a;^ = 77^ = a;„ < and x„ G [-C, C]. For Tea and 
xl < ivn + Vn-if < {{a - l)-i/2 + a-i/2)2 < ^2 g 

Case I. Consider the case that Forecaster uses the strategy J^i, Skeptic uses 
T, and Reality uses TZi. 

Consider the capital K.^^''^'^^. By the definition of i]n in (IT^ . M^rjn < 
and 77,^ — w„ = for all n. Hence 
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for all n and /C^^'^'^^ is monotone non- increasing in this case. Then 
We will prove 

^^i,Qi,Ki ^ jcf^2,Q,TZ2 and /C^i^S^^^i = ^f^^'Q'^-^ (14) 

for all n. Then by prudence of 7^ and Q, the three capitals K.'^^''^''^^ , K,^^'^^'^^ 
and JC^^'^^'^^ are all non- negative. Hence they are bounded by p^ . 

When Forecaster uses J^i, ^ and X]n '^n = ^he property 

pT|) of P and the boundedness of /C^^''^'^^ ^ ^'^m) = X^n In does not 

converge. Note that 77„ converges to 0. 

Case II. Consider the case that Forecaster uses J^2, Skeptic Q and Reality 
Tl2- In this case 

n n 

fc=l fc=l 

hence it converges. By the property (ITOl) of Q and the boundedness of IC^^'^'^^ , 
we have (-770,1:^1, -771,^2, ...)€£'. 

Case III. Consider the case that Forecaster uses J^2, Skeptic Q and Reality 
7^3. In this case 

n n n 

Sn = '^Xn= ^{-Vk - Vk-l) = ^^(^ r]k) + T]n 
k=l k=l k=l 

hence it does not converge. Similarly, by the property (|10p of Q and the bound- 
edness of /C^^'^'''^^, we have (— 7701 ^1, —111^^2, ■ ■ ■) ^ D. This is a contradiction. 
It remains to show the equations (fH)) . that is, 

for all n. We prove this by induction. For n = 0, we have IC^^^^^'''^^ = 
/C^2.Q,TC2 ^ /C^i,S2,7Ji ^ /C^2,Q,7J3 ^ 1. Suppose /C;^if ^'^^ = /C^^f ^"^^ and 

_r-^2,Q,K3 

This completes the proof. □ 
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Therefore we are interested in when we can determine the convergence of 
X^n^" ^'^"^ Sn^raj ^^'^ whcn wc Can not. From Theorem 13.11 com- 

bined with Proposition 12. 2[ we can easily prove the following relations. 

Corollary 3.3. In BFQH Skeptic can force the following events: 

(i) "^n converges and < oo =^ ^" converges, 

(ii) m„ does not converge and w„ < oo =^ a;„ does not converge, 

(iii) ^„ m„ converges and = cx) => x„ does not converge. 

(iv) x„ converges => ( m„ converges and u„ < oo) or 

(limsup^ mfc = oo, liminf ^ mfc = -oo and 5^«„ = oo)) (15) 

" fe=l " n 

(i) and (ii) follow from (^) of Theorem l3.1l (iii) follows from (<^=) of Theorem 
13.11 and (iv) follows from the fact that if Skeptic can force an event E in the 
bounded forecasting game, then he can force E in BFQH. 

In the classical three-series theorem, the second case of the right-hand side of 
(|15p is eliminated by the assumption of independence of the random variables. 
In view of Gilat's counter example, it seems that a simple general statement for 
the game-theoretic framework can not be given for this case. However in some 
special cases, where the behaviors of m„ and w„ are simple, we can give 
definite statements. In Corollarv l3.4l and Corollarv 13.51 we discuss such cases. 

One simple case is that Reality's move Xn is restricted to be non-negative. 

Corollary 3.4. (One-sided BFQH) Consider the following special 
[0, C] in BFQH. Then Skeptic can force 

TO„ converges and w„ converges <4> Xn converges 

n n n 

This corollary easily follows because by coherence m„ > and and rUn 
is a non-negative series, which eliminates the second case on the right-hand side 
of Q. 

We now consider the case that the move space of Reality is restricted to be 
a set of three points (trinomial game, cf. [IB])- This case will play an essential 
role in Section [1751 Indeed the counter examples to SLLN in Section 4.3 of [23] 
and Section 7 of [T^ are constructed as probability distributions on a set of 
three points. 

Corollary 3.5. (Trinomial Game) We consider the following special case in 
BFQH: 

rUn = 0, Vn e [0, 1], Xn € {0, ±1}. 

Then Skeptic can force 

u„ < oo Xn = for all but finite n. 

n 
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Proof. This follows from Theorem 13.11 because for Xn G {0,±1}, X^n '^o'^" 
verges if and only if a;„ = for all but finite n. □ 

For the rest of this section we again consider the coherence of BFQH men- 
tioned just after Theorem 13.11 in relation to Corollary 13.51 The coherence can 
also be proved by a direct calculation as follows. If 1^ > then Reality an- 
nounces Xn = nin- Then Mn{Xn — 1Tln) + Vn{{Xn — " Vn) < 0. SuppOSe 

Vn < 0. If m„ > 1^ then Reality announces a;„ = —C. Then 

Mn{Xn -rUn) + Vn{{Xn - m„)^ - U„) 

<M„{-C - m„) + Vnii-C - TTinf - {C^ - ml)) 
= - M„(C + m„) + 2Km„(C -I- m„) 
= (2Km„ - M„)(C + m„) < 0. 

If T^i < and m„ < then Reality announces Xn ~ C. The calculation is 
the same as above. This fact is important for our argument in Section 14. 3[ so 
we state it as a proposition. 

Proposition 3.6. BFQH remains coherent even with the restriction Xn S 
{m„,±C}. 

4 The rate of convergence of SLLN 

In this section we consider the rate of convergence of SLLN in the usual un- 
bounded forecasting game with quadratic hedge. 

Unbounded Forecasting 
Players: Forecaster, Skeptic, Reality 
Protocol: 
/Co = l. 

FORn = 1,2,...: 

Forecaster announces m„ £ R and > 0. 
Skeptic announces M„ S M and > 0. 
Reality announces Xn G R. 

JCn /C„_i -I- Mn{Xn - TUn) + Vn{{Xn - m„)2 - Vn)- 

Collateral Duties: Skeptic must keep /C„ non- negative. Reality 
must keep /C„ from tending to infinity. 

As in the last section we use the notation y„ = T^l^^ii^k — f^k), An = 
Sfc=i ^fe ~ liniji An- In many cases we assume that to„ = for all n 

without loss of generality. 

4.1 Motivation 

The rate of convergence of SLLN for i.i.d. case was completely solved by LIL of 
Hartman and Wintncr We refer to [23] . 
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Theorem 4.1 (Hartman-Wintner's law of the iterated logarithm). Let {Xn} 
be a sequence of independent identically distributed random variables with zero 
mean and finite variance . We put Sn ~ X)fc=i"^'^' '^^ ~ (2Hloglogn)"'^/^. 
Then 

lim sup S'„/a„ — a a.s., lim inf S'„/a„ = —a a.s. 

The theorem was a generalization of the case of binary sequences by Khinchin 
|11] . If we drop the condition of i.i.d., we need some additional conditions. 

Theorem 4.2 (Kolmogorov |12|). Let {Xn} be a sequence of independent ran- 
dom variables with zero means and finite variances. Put cr^ — VarXn and 
Bn — X]fc=i "'fc- Suppose Bn — > OO. Suppose also that there exists a sequence of 
positive constants {M„} such that 



Bn 



^ log log Bn 



1/2N 



and 

\Xn\< Mn a.s. 

Then 

lim sup -TTTT = 1 a.s. 

^ (2B„loglogB„)i/2 



Some other condition for LIL than in Theorem 14.21 is given in 29 and LIL 
for martingales are discussed for example in |28j and For game-theoretic 
LIL see Chapter 5 of [24, and a recent paper by Takazawa |25j . 

In game-theoretic probability we can not assume that the sequence {xn} an- 
nounced by Reality is i.i.d. Furthermore we can not assume the existence of Mn 
either such that |x„| < Mn- From now on we consider the rate of convergence 
of SLLN in game-theoretic probability. In the view point of measure-theoretic 
probability it is the rate of convergence of SLLN in a non-identical case. 

4.2 Results on convergences in unbounded forecasting 

Here we derive several results on the rate of convergence of Kronecker's lemma 
and hence the strong law of large numbers. 

Lemma 4.3. In the unbounded forecasting Skeptic can force 

n 

u„ < OO ^^(xfc — mk) converges. 

n k=l 

Skeptic can not force 



^^(xfe — TOfe) converges => u„ < oo. 



k=l 
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Proof. The proof of the first statement is exactly the same as the proof of (=>) 
in Tlieorem l3.1l For the second statement, consider Reality's strategy Xn — ran, 
Vn. Then /C„ = /Cq — X]fc=i ^k^k and clearly Skeptic has no control over i)„'s 
and hence can not achieve ''^n < oo. □ 

Theorem 4.4. Let g be a positive increasing function. In the unbounded fore- 
casting Skeptic can force 

Proof. We assume that m„ = for all n without loss of generality. We consider 
the capital process 

k<n vgi^k) 

The compensator of is 

n 

Vk 

If Boo < oo, then Wn converges by Lemma 4.6 and Lemma 4.7 in [24]. □ 

Corollary 4.5. Let g be a positive increasing function on [0, oo) with g{oo) = 
oo . Skeptic can force 

> Vn — OO and > , , , < oo => ~ , : > (J. (17) 



Proof. This follows easily from Theorem 14.41 and Kronecker's lemma. □ 

In some cases we can drop X)n^"/5(^") °° from (fT7| . The following is 
an example. 

Corollary 4.6. Let g be a positive increasing function such that 

dx < oo. 



gix) 

Then Skeptic can force 

. ^ Ek<ni^k~mk) 
Aac = OO ^ > 0. 

Proof. It suffices to show that °° when Aoo — oo. This holds 

because 

/■oo 1 °° rA„ 1 oo 

Jo 9{x) ^Ja„^^9{x) ~ ^^9{An) 

□ 
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Example 4.7. We write In' to mean the function such that In' (a;) — ln(ln* ^(x)) 



and XtP [x] — x defined recursively. Let 



g,{x) (Hln^) xln'(x). 

In other words, 

go{x) — x^, gi{x) — x{\nx)'^, 92{x) — a;lna;(lnlna;)^. 

Then ^ ^ 

-dx = — - 



gi{x) In' a;' 

Hence Skeptic can force 

■ Efc<„(a^fe-™fe) 

Ar^, = OO 



for all i. 

For example, consider the special case Vn = v. Then A,^ — oo is automatic 
and with 52 (x) above we have X]fe<n(^'s ~ "^fc)/ {Vnlnn In In n) — > 0. 



Note that it does not follow X]/c<n('^fe ~ rnk)/Vn Inn In Inn 0. This is 
because — ; — 1—, — = 00. However this does not mean that Theorem 14.41 is 

n Inn In Inn ' ' 

a weaker result compared to LIL. Recall that we are considering a non-identical 
case in the measure-theoretic point of view. In fact Theorem 14. 41 is strict as we 
will see in the next subsection. 

We now apply Corollary 14.51 to the coin-tossing game. We first show that 
the coin-tossing game is a special case of BFQH. Restrict Xn G {0, 1}, m„ = 
Pn e [0, 1] and Vn = p„(l - _p„) in BFQH. For M,V eR, p e [0, 1], consider 

M{x -p) + V{{x ~ pf - p{l - p)) 

For both a; = and a; = 1 we have {x—p)'^—p{l—p) = {x~p){l~2p). Therefore 

M{x-p) + V{{x-pf -p{l-p)) = M{x-p) + V{x-p){l - 2p) 

= {M + V{l~2p)){x-p). 

By considering Skeptic's move Af„ -I- Vn(l — 2p„), we see that BFQH for a;„ g 
{0, 1} reduces to the coin-tossing game. Therefore from CoroUarv 14.61 applied 
to the coin-tossing game, we have the following result. 

Proposition 4.8. Let g be a positive increasing function such that '^^dx < 
00. Let An = '}2k=iPk- Then in the coin-tossing game Skeptic can force 

Aoo = 00 ~ > 0. 

VgiAn) 
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Proof. Let An = J2k=iPki'^ -Pk)- Then A„ < An- If ^oo = ^oo = oo, then by 
Corollary 14.61 Skeptic can force 



y/giAn) yff(XO V ai^n) 

On the other hand, if A^o < Aoo — oo, then Skeptic can force the convergence 
of Yl^ni^ri - Pn)- Howcver in this case J2k<ni^k - Pk)/\/9{An) 0, because 
g{co) = 00. □ 

Note that Proposition 14.81 strengthens (ii) of Remark 12.51 which only states 

T Y.k<niXk-Pk) T,k<n^k 

Aoo = 00 ^ = 1^-0. 

An l^k<nPk 

4.3 Results on divergence in unbounded forecasting 

As the converse to the convergence result in the previous subsection, we will 
prove that Skeptic can not force the convergence on the right-hand side of (|16p 
when J2n '^n/ g{An) = 00. The novelty in our approach is that in order to prove 
this fact we use deterministic strategies of Reality. 

We formulate Reality's strategies and introduce the notion of compliance of 
Reality with an event. We propose to use the term "compliance" for Reality's 
strategies and reserve the word "forcing" to Skeptic for clarity of our arguments. 
Skeptic's forcing an event E means that Reality has to move so that E happens 
in order that Skeptic's capital stays bounded. In contrast Reality's forcing an 
event E means intuitively that Reality can move so that E happens and Skeptic's 
capital stays bounded irrespective of the moves of Forecaster and Skeptic. In 
other words Reality conforms to E and adapts as requested. 

For notational simplicity, as in the multi-dimensional bounded forecasting 
game, write /i„ = (r7i„,w„), m„ = (M„,V^). Consider Reality's strategy TZ = 
{Ti-n}n>i which determines Reality's move Xn based on the moves Hk^^^k, k < n, 
of Forecaster and Skeptic: 

a;„ = 7^„(/ll,Tnl, . . . ,^„,m„), n>l. 

Definition 4.9. By a strategy TZ Reality complies with the event E d if 

(i) irrespective of the moves /i„,m„, n>\, of Forecaster and Skeptic, both 
observing their collateral duties, it holds that 

(/ii,7?.i(/ii,Tni),^2,72.2(MiiTni-A'2,Tn2),...) € E, 

and 

(ii) sup„ Kn < 00 . 
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We say that by TZ Reality strongly complies with E if the supremum in (ii) is 
uniformly bounded from above by 1 = JCq, i-e., /C„ < 1 irrespective of the moves 
of Forecaster and Skeptic, both observing their collateral duties. 

We simply say that Reality (strongly) complies with the event E d 'E. if there 
exists a strategy TZ such that by TZ Reality (strongly) complies with E. 

Concerning the notion of compliance we prove the following fundamental 
proposition. 

Proposition 4.10. In the unbounded forecasting, if Skeptic can force an event 
E, then Reality strongly complies with E . 

Remark 4.11. As seen in the proof of this proposition below, the statement holds 
not only for the unbounded forecasting, but for more general protocols of game- 
theoretic probability, including the trinomial game. 

The idea of the proof is as follows. First of all Reality needs to prevent the 
capital from tending to infinity. By coherence this is possible for Reality. Next 
the path must be in E. Since Skeptic can force Skeptic has the strategy such 
that if the path is not in E, then the capital goes to infinity. It follows that if 
the capital does not tend to infinity, then the path is in E. Then all Reality has 
to do is to prevent the capital of the strategy from tending to infinity. Again by 
coherence this is possible for Reality. Can Reality prevent the capitals of two 
strategies from tending to infinity? It is possible by considering the strategy 
that is the average of two strategies. In other words Reality's strategy can 
be constructed by considering a single sufficiently powerful strategy of Skeptic. 
Furthermore Reality's strategy can be deterministic. To make the strategy 
"strongly" comply we need a more precise argument as is in the proof. 

Such an argument is commonly used in algorithmic randomness. Especially 
some examples of random sets are sets on which a single sufficiently powerful 
(super)martingale fails. See [ini [SJ [THl HSl [T7]- One way to obtain a set on 
which the (super)martingale fails is to choose the leftmost non-ascending path 
in binary sequences. This choice corresponds to the coherence in game-theoretic 
probability. Although the constructed random sets may not be computable in 
general, it can be constructed deterministically by the (super)martingale. 

We set up some more notation for clarity. When the moves of all the players 
are individually specified we write Skeptic's capital as 



We now give a proof of Proposition [47101 

Proof. Since Skeptic can force E, there exists Skeptic's strategy V such that 



JCn[{^J.k,mk,Xk)l^l% (/Co = 1). 



In this notation Skeptic's capital under a strategy V is written as 




A:„[(/i/c,^/c,a;fe)Li]- 



lim sup /C^ 



< oo {jln.Xn). 



'n=l 



oo 



e E. 



(18) 



n 
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First we give a strategy TZ of Reality such that /C„ is uniformly bounded from 
above by 1 + e, where e > is arbitrarily fixed. 

Consider Reality's move at the first round n — 1 after Forecaster's move fii = 
{mi,vi) and Reality's move mi = (Mi,Vi) were announced. Write 7-'i(/xi) = 
Vi{{mi,vi)) = {Mi',Vi'), which is the move of the strategy V at the first 
round. Let a = 1/(1 + e) and let 

mi = {Mi,Vi) = (l-a)(Mf,Fi^) + a(Mi,yi). 

Because of coherence, Reality can (deterministically) choose xi such that 

/Ci[/ii,mi,xi] = (1 - Q;)/Ci[^i,7'i(/ii),a;i] + Q;/Ci[^i,mi,a;i] 
</Co = l. 

Since both A^i [/ii, 7'i(/ii), xi], /Ci [^i, mi, 2:1] are non-negative, it follows that 

ICi[fii,Vi{fii),xi] < — = ^— /Ci[^i,mi,xi] < - = 1 + e. 

1 — a e a 

We now make an inductive argument. Suppose that Reality could determin- 
istically choose xi, . . . , Xn-i such that 

K-n-l[{^ik.Vk,xu)lZl] < ]Cn-i[{tJik,mk,xu)lZl] < 1 + e- 

As in the first round define 

m„ = (M„, K) = (1 - a)(M^, V^) + a(M„, V;), 

where (Af„, — m„ is the actual move announced by Skeptic and (M^, Vl^) 
is the move of strategy V . By coherence. Reality can now choose a;„ such that 

M„(X„ - m„) -I- Vn{{Xn - ninf - Vn) < 0. 

Then 

ICn[i^ik,mk,Xk)k=l] = (1 - a)A:„[(/ifc,7'fc,a;fc)fc^i] -I- alCn[ink,mk,Xk)k=i] 

< ICn-l[{Hk,mk, Xk)kZl] 

< 1. (19) 

Thus as in the first round 

ICn[{f^k,Vk,Xk)k=i]<^^, ICn[{f^k,mk,Xk)l=i]<l + e. (20) 

By and the first term of (PO]) . (i) of Definition l4.9l is satisfied. By the second 
term of ICn[{pk,y^k,Xk)^^i] is uniformly bounded from above by 1 -I- e. 

It remains to show that we can let e = 0. We argue as follows. By coherence, 
Reality can always choose Xn such that Af„(a;„ — m„)H-l/„((a;„— m„)^— w„) < 0. 
In the unbounded forecasting, unless {Mn,Vn) — (0,0), Reality can choose Xn 
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such that this inequahty is strict. ReaUty will look for the first time n = 
no such that {Mn,Vn) ^ (0,0). At round no Reality chooses Xng such that 
^no [(^fc: ^k, a;fc)feli] < 1- For n = 1, . . . , no — 1, Reality chooses Xn such that 
l^n — 0- Now define a = /Cno [(/^fei Trife, a;fe)fc!Li] and after the round no Reality 
follows the strategy ensuring /C„ < 1, n > no, as in (IT^ . On the other hand, if 
there is no such no, then Reality keeps choosing a;„ such that /C^ < 0. In this 
case /C„[(/ifc,mfe, a;fc)5J^2^] = 1 for all n and also (i) of Definition 14.91 is satisfied 
by CH]). □ 

We now state the following theorem. 

Theorem 4.12. Let g : R — )■ R 6e a positive increasing function. Then in the 
unbounded forecasting Reality strongly complies with the event 



/ r A \ = OO ^ 

4" gi^n) 



Y.k<ni^k-mk) 



does not converge. 



(21) 



As an immediate consequence of this theorem we have the following corollary. 

Corollary 4.13. Let 5 : M — >■ R 6e a positive increasing function. Let Ei he 
any event depending only on ui,W2, • ■ • , such that Ei n {X^n g(X ) ~ 
In the unbounded forecasting Skeptic can not force 



El 



converges. 



^9{An) 

For proving Theorem 14. 121 we prove two technical lemmas. 

Lemma 4.14. Let {yn} be a sequence of reals and let {gn} be a non- decreasing 
sequence of positive reals. If iJ2k<nyk)/ 9n converges tod, then \yn/gn\ < MI + 1 
for all but finite n. 

Proof. First note that 



Vn 


< 


Vn gn-A 


+ 




< \d\ + 


yn 9n-l\ 


gn 




gn V gn J 








gn V gn J 



Therefore it suffices to show that 

Vn 

gn 



d 1 



gn-i 



< 1 



for all sufficiently large n. 

Let e be such that < e < -j. Then there exists N such that 



n> N 



En 
k=l Vk 



-d 



< e. 
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It follows that, for all n — 1 > iV, 

9n V 9n J 

^ yjt_ Efc=l Vk ^ L _ 9n-l\ 
~ 9n gn-1 \ 9n ) 

Qn 9n-l 

□ 

Lemma 4.15. Let {a„} be a sequence of positive reals. Then there exists a 
sequence {£«} of positive reals such that 

(i) e„ is determined only by ai, - ■ ■ ,an, 

(ii) e„a„ < 1, 

(iii) J2n On = oo implies J2n ^^'^n = oo and e„ 0. 
Proof. We define e„ as follows. 

(PI) Let n = 6 = c = 1. 

(P2) If 2^''a„ > 1 then let e„ = l/fln, otherwise e„ = 2^'', i.e. let e,i = 
min(l/a",2-''). 

(P3) If X;Lc ^ 1 then let & = 6 + 1, and c = n + 1. 

(P4) Let n = n + 1 and goto (P2). 

It is clear that (i) and (ii) are satisfied. We shall prove that (iii) is satisfied. 
Suppose that J2n — oo. 

We claim that '^k=c^kO-k > 1 in (P3) for infinitely many times. Otherwise, 
6 and c do not change from some point. Also, in view of 

n 

Cfca/c < 1 ^ enttn < 1 ^ e„ = e"'', 

k—c 

€n = 2^^ for all but finite n. It follows that X]fc>c ^ ''^fe < 1- This contradicts 
to the fact that J2n '^n — oo. 

Then there exists an increasing sequence {ci} such that X]fc=ci+i ^kO,k > 1- 
It follows that 

Since b goes to infinity and e„ < 2^^ in (P2), we have e„ — 0. □ 

We are now ready to give a proof of Theorem 14. 121 Before starting a formal 
proof, we discuss the idea of the proof. By Proposition 14.101 we wish Skeptic 
could force the divergence. As we saw in the Lemma l4?3l however, it is not easy 



En — 1 
k=l Vk 



gn-1 



e < 



En 
k=iyk 



-d 



9n 



En — 1 
fc=l Vk 

gn-1 



- d 



e < 3e. 
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for Skeptic to force the divergence in the unbounded forecasting. So we use the 
bounded forecasting, more precisely, the game restricted to three points as in 
Proposition 13.61 

It is suffices to show that Skeptic can force (PT|) in the restricted protocol by 
the following reason. Consider the following three statements. 

(i) Skeptic can force the event in the restricted protocol. 

(ii) Reality strongly complies with the event in the restricted protocol. 

(iii) Reality strongly complies with the event in the unbounded forecasting. 

The implication (ii)=4>(iii) holds by the following simple argument. Suppose 
that Reality strongly complies with it in the restricted protocol. Then Reality 
can use the same strategy in the unrestricted protocol or in the unbounded 
forecasting. 

The implication (i)^(ii) follows from Proposition 14.101 Note that the re- 
stricted protocol Skeptic can use a strategy with Vn < in order to force (PT|) . 
In the unrestricted protocol this is not allowed. It is Reality who refers this 
Skeptic's strategy to make her strategy. 

To prove (i), we use the trinomial game (cf. Corollarv 13.51 and Proposition 
13.61) . In other words we construct a reduction from the restricted protocol 
to the trinomial game. Then since Skeptic can force the divergence in the 
trinomial game, Skeptic can also force the divergence in the restricted protocol. 
This argument of the reduction is often seen in computability theory [3TJ (35] , 
complexity theory [1] and algorithmic randomness [H [19] . 

We now give a formal proof. 

Proof of Theorem \4-.12\ For notational simplicity we write yn — Xn — nrim gn = 
yJg{An) and z„ — yn/ gn- Let e„ > be such that is a sequence of Lemma 
14. 151 for a„ = restrict Reality's moves as e„2:„ € {0, ±1}. The capital 

process is 

K.n ^ ICn-l + MniXn - TTln) + Vn{{Xn - mnf - W„) 

r _L T\,r 9n , 9{An) ( , .2 2 Vn\ 

= AL„-1 + Mn — e„z„ + Vn — 5 — (enZ„) - £„ . . . 

= K.n-1 + M:^{x[, - m'J + V;:(« - m'J^ - <). 

Then /C„ can be seen as a capital process in Corollary 13.51 where Forecaster 
announces mj^ — and w'^ — ) ' Skeptic announces = M„|^ and 

= ^4"'' ^^'^ Reality announces x'n = £nZn G {0, ±1}. Note that v'n € [0, 1] 
because of the definition of e„. Since |e„z„| is bounded, Vn can also be negative 
in the restricted protocol. Hence, by Corollarv 13. 5[ Skeptic can force that 

5Z r = 00 e„z„ = ±1 for infinitely many n 
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and Reality can choose her move such that Kn < /Cn-i < • • • < A^o = 1 for ah 
n. 

By the definition of e„ we have Y.n ^(aIJ = oo ^ Y^n ^"^(31) = 
thermore If e„|z„| = 1 for infinitely many n then |z„| = l/e„ for infinitely many 
n and sup„ \zn\ = oo by — and e„ 0. It follows that (X]fc<n yk)/gn does 
not converge by Lemma [4. 141 □ 

We can summarize the results of this section as follows. Consider a game in 
which Skeptic wins when converges. If both players do the best, 

then the winner depends only on whether g{^77 < oo or not. In other words 
in order to make it converge we need a function g such that ^„ g(3^ < oo. As 
we have already seen in Example 14.71 this g grows faster than n log log n that 
appears in LIL. 

A measure-theoretic interpretation is as follows. Let to„ and w„ be the 
conditional mean and the conditional variance. Under any probability measure, 
if N < 00, then V^,^^ ^k-mk converges a.s. Now consider the case 

) ~ There exists a probability measure, such that if J2n g(A ) ~ °^ 
then YlkKn ^/ 7™'° does not converge a.s. 

5 Rate of convergence of SLLN under a general 
hedge 

Theorem l4.4l can be seen as an extension of SLLN. In contrast Kumon, Takemura 
and Takeuchi proved another extension. Furthermore both extensions have 
similar forms. To clarify a relation between these two extensions we consider a 
protocol with a general hedge. 

Unbounded Forecasting with General Hedge (UFGH) 
Parameters: A single hedge /i : M M 
Players: Forecaster, Skeptic, Reality 
Protocol: 
/Co = l. 

FOR n = 1,2,...: 

Forecaster announces m„ G M and w„ > 0. 
Skeptic announces M„ S M and U„ > 0. 
Reality announces G K. 

/C„ := K.n-1 + Mn{x„ - TO„) + Vn{h{x„ - TO„) - V„) . 

Collateral Duties: Skeptic must keep /C„ non-negative. Reality 
must keep /C„ from tending to infinity. 

We assume a few conditions for /i : M M. 
(AO) h{x) = h{\x\) > 0. 

(Al) For some c > 0, h{x)/x is monotone increasing for x > c. 
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(A2) For some c > 0, h{x)/x'^ is monotone decreasing for x > c. 

In (A2) we are considering hedges weaker than the quadratic hedge, which 
is our main interest in this section. The imphcations of hedges stronger than 
the quadratic hedge to the rate of SLLN are investigated by recent papers of 
Takazawa {[IB], [IS]). 



5.1 The case of convergence 

The following theorem is a generalization of Theorem 14.41 to the general hedge. 

Theorem 5.1. Suppose that h satisfies (A0)-(A2) and that g is a positive in- 
creasing function. Then in UFGH Skeptic can force 



EVn \ ^ 



9{An) ' h-^og{Ak) 



converges. 



Corollary 5.2. In UFGH Skeptic can force 

n 

Vn < oo ^^(xfc — TOfc) converges. 

n k=l 

and 



Vn — oo and —- 



< oo 



g{An) h-^og{An) 

This corollary follows from the theorem and Kronecker's lemma. 
Notice that Theorem 15.11 with h{x) — x"^ implies Theorem 14.41 In contrast 
Theorem 15.11 with g{x) — h{x/v)^ m„ — and Vn — v implies Theorem 3.1 of 

m- 

The rest of this section is devoted to a proof of Theorem 15.11 The proof is 
just a straightforward combination of that of Theorem l4.4l and that of Theorem 
3.1 of [13]. Note that, without loss of generality, we assume that m„ = for all 
n. 

Remark 5.3. We can assume that c in (Al) and (A2) are and h(x) = x^ 
for \x\ < 1 by the following reason. Let c be such that h[x)/x is monotone 
increasing and h{x)/x'^ is monotone decreasing for x > c. Let 



hoix) 



^x if \x\ < c 
h{x) otherwise. 



Then Hq satisfies (Al) and (A2) for a; > 0. It also follows that h ^(y) > is 
defined for y > 0. 
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By this remark we assume that c in (Al) and (A2) are 0. Let 
hi{x) 



if \x\ < 1 
h{h^^{l) ■ x) otherwise. 



Then hi satisfies (A0)-(A2). Furthermore, for large x, hi{x) — h{h^^{l) -x) — y 
and h^^{y) = h^^{l) ■ x = h^^{l) ■ h^^{y). Then the convergence does not 
depend on the use of h and hi. We also have h{x)/x'^ < 1. 

Lemma 5.4. In UFGH Skeptic can force 

Proof. Consider the strategies such that 

/Co = D, Mn = 0, 14 ^ 



as long as /C„ is non- negative for all Xn where D is a natural number. If 
J2n Vn/g{An) < D then the capital process /C„ is 



hjxk) - Vk 
9iAk) • 



fc=i 

If Y^n'^n/giAi) < oo and J^nH^n)/ 9{Ai) = oo, then lim„/C„ = oo. This 
proves the lemma. □ 

Lemma 5.5. Under the conditions (AO), In UFGH Skeptic can force 

/or onZy finitely many n. 
Proof. Notice that 

\Xn\ > h-^og{An) 

implies 

giAn) - 

Then the result follows from Lemma [5^ □ 
Lemma 5.6. Under the conditions (AO) and (A2), In UFGH Skeptic can force 

2 

EVn \ ^ X 



28 



Proof. By Lemma [5.51 we only consider the case Xn < h ^ o g{An)- By (A2), 
h{x)/x^ is monotone decreasing for x > 0. Hence 

h(xn) hoh~^ ogjAn) 
xl - (/i-io5(A„))2- 

It follows that 



Then the result follows from Lemma WM □ 



Lemma 5.7. Let e be such that < e < j. Under the conditions (AO) and 
(Al), for all X and n 

" \x\ h(x) — v„ 1 



giAn) ~ h-^og{An) giAn) ~ 2 

Proof By (Al), for x > o g{A„), 

h{x) ^ 



X h iog(A„)' 
It follows that 

h{x) X ^ p 



g{An) h~^og{An) 
For < X < o g{An), 

h{x) X 

g{An) h-^og{An) 

Therefore for all a; > we have 

h{x) — Vn X 



> -1. 



> -1 



g{An) h-^og{An) g{An) 



Lemma 5.8. Under the conditions (A0)-(A2), in UFGH Skeptic can force 



□ 



converges. 



Proof. We consider the following two strategies simultaneously such that JCq — 1 
and ^ ^ 

Mn — ±e/C„_i- — , . , , Vn = e/C„_i— r-— - 

h ^ o g{An) g[A-n) 

if Vn/g{An) < 1 and M„ = Vn = otherwise, where 0<e<l/4is fixed. We 
denote the corresponding two capital processes as /C^ . Since ^„ w„/g(^„) < oo, 
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we have Vn/g{An) < 1 for all but finite n. Hence we can assume that the 
inequality in the right-side hand of ([22)) holds for all n. Then 



h o g[An) g(An) 2 

Hence K.^ > for all n. By the game-theoretic martingale convergence theorem, 
both K.^ converge to a non-negative finite value. 

By the inequality t > log(l + t)>t-t^ for all t > -1/2, 

E/ ±x„ hjxk) - Vk 



±Xk , h{xk)-Vk\ 2^ f , h{xk)-Vk^'^ 



k=l 

>log/C, 

n 

Notice that each of the following infinite sums is finite; 



By the inequality 

(ai -I h a™)^ < m{a\ H h a^) 



we have 



E 



±Xk , - Ufe^ ^ 



< CX3. 



It follows that 



-^\h-^og{Ak) g{Ak) 

sup > < OO. 

n t^,h~'°9iAk) 

Hence both log/C^ converge to a finite value. Therefore we obtain the desired 
result. □ 

5.2 The case of divergence 

We now consider the case of divergence of J2n '"n/g{An). 

Theorem 5.9. Suppose that h satisfies (A0)-(A2) and that g is a positive in- 
creasing function. Then in UFGH Reality strongly complies with 

n 

„ = °" ^ E h-^ogiAk) ^"'^ 
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Proof. The proof is similar to that of Theorem 14.121 We will consider the 
protocol such that Reality's move is restricted for each n. We shall show that 
Skeptic can force and Reality strongly complies with (1^51) in this protocol. 

Let y„ ^ Xn ~ m„, gn = o g{An) and z„ = t/„/5„. We restrict Zn G 
{0,±1}. If |z„| = 1 then h{yn) = h{g^) = g(^„). If z„ = then /i(y„) = h(Q) = 
0. In any cases we have = g(A„)z^. 

The capital process is 



/C„ = K,n-l + M„g„Zn + VngiAn) z 



2 



giAn) 

By Corollary 13.51 Skeptic can force 

^ = oo => |z„| = 1 for infinitely many n 

and Reality can choose a;„ such that /C„ < /C„_i and hence sup„/C„ < 1. If 
|z„| = 1 for infinitely many n, then yn/g-n does not converge. Hence Skeptic 
can force (l23l). □ 



We will show that ^f=ilg(^"'°^ does not converge under the additional 
condition h{xy) = h{x)h(y) for all x,y > 0. However this condition implies 
h{x) = x'"" and h^^{x) — x^/*" for x > 0. Taking into account of (Al)-(A2) we 
have 1 < r < 2. 

Theorem 5.10. Let h{x) = x^ where 1 < r < 2 and g he a positive increasing 
function. Then Reality strongly complies with 

> , , , — oo — 7-^^ — — — does not converqe. 

Proo/. Let y„ = a;„ - m„, 5„ = ft,"^ o g{An) and z„ = y-algn- This time let {e„} 
be such that /i(e„) is a sequence of Lemma [4. 151 for ^ . We can assume that 
< Cn < 1 for all 71. We restrict e„z„ e {0, ±1}. Then we have 

h{yn) - Vn ^h(gn)h{zn) - v„ = ■ " /i(e„z„) - /i(e„) 



(e„z„) - /i(e„)- 



Hence the capital process is 

K-n = ICn-l + Mnyn + Vn{h{yn) - Vn) 



= K-n-i + Mn — e„z„ + Vn—, — 7 (e„z„) - /i(e„) 



By Corollary [331 Skeptic can force 

$Z ^(^w) / r \ = °o =^ f^tiknl ~ 1 for infinitely many n 
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and Reality can choose Xn such that Kn < A^n-i and hence /C„ < 1. The rest 
of the proof is the same as that of Theorem 14.121 □ 

One may think that this result contradicts Marcinkiewicz-Zygmund strong 
law, which says that for i.i.d. random variables {a;„} with i?|a;„|'" < oo for 
< r < 2 and Exn — when 1 < r < 2, n^^^^ i^^^i Xk) — >■ as n — oo a.s. 

For example let m„ = 0, w„ = u for all n and g{x) = x/v. Then gn = 
{g{An)Y/'^ = n}/'' and e„z„ = e„y„/5„ = e„a;„/ni/''. By e„z„ e {0, ±1} we 
have Xn € {0,±n^^^ /en}- Since e„ — ?> 0, we have n^^^/en ^ oo as n — ?> oo. 
It follows that the restrictions of x„ are not the same. Thus this is not a 
contradiction. 

Theorem 15.11 and Theorem 15.91 give the rate of convergence of SLLN under 
a general hedge in a non-identical case. 

6 Discussion 

In the classical three-series theorem, under the assumption of independence of 
random variables, the necessary and sufficient condition for the convergence of 
a random series is given by the convergence of three series: truncation proba- 
bilities, truncated expected values and truncated variances. On the other-hand 
Gilat's counter example shows that the necessity of the convergence of three 
series can not hold for martingales. Therefore a question of interest is to specify 
some conditions, other than the independence, under which the convergence of 
a random series implies the convergence of three series. We have shown that 
under the convergence of a random series, the divergence of truncated means 
can only occur as two-sided unbounded oscillation. We also gave some sim- 
ple conditions of convergences of truncated means and variances, but stronger 
results are desirable. 

We proposed the notion of compliance concerning Reality's deterministic 
strategy. We showed that a good deterministic strategy of Reality can be auto- 
matically constructed by using a good strategy for Skeptic as a "surrogate" . In 
Definition 14 . 91 we made a distinction between compliance and strong compliance 
concerning Reality's deterministic strategy. Further study is needed to clarify 
the difference between these definitions. 

We gave the precise limit of the rate of convergence of SLLN with the 
quadratic hedge as well as more general weaker hedges. According to it the 
rate of convergence for a non-identical case may be slower than for the i.i.d. 
case. We believe that game-theoretic probability is a powerful tool for analysis 
of such a non-identical case. 
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